Population-Based Survey Experiments A HyBrid MEtHodology for tHE SociAl SciEncES

Approaches to scientific knowledge are a bit like rabid sports rivals; often they cannot say anything nice about their own team without simulta­ neously disparaging the other side. At some level, they know these intense rivalries would not exist if the other team were not a worthy contender, but the positive aspects of the other side are seldom acknowledged. likewise, empirical social scientists tend to develop expertise either in large-scale observational methods such as survey research, or in laboratorybased experimental approaches. They then spend the rest of their careers defending their choice of that particular approach in virtually everything they publish. Each time we submit a journal article, we rehearse all of the advantages of our own methodological choice, briefly mention its weak­ nesses, and make the case in no uncertain terms that what we have spent our time on is worthwhile. go team! competition among methodological approaches is certainly implied, even if it is not explicitly stated. We do our best to defend our own ingroup by stressing the importance of internal validity if we have produced an experiment, or external validity if we have completed an observational study. fortunately, this caricature is gradually becoming less accurate, both in terms of its characterization of researchers—an increasing number of whom are trained in multiple methods—and in terms of how methodolo­ gists are focusing their attention. Although there are still many survey re­ searchers working on improving their particular method, and many exper­ imentalists focused on developing innovative experimental techniques, there are also methodologists paying specific attention to the problem of integrating results from experimental and observational studies. for the most part, these approaches involve applying complex statistical models to estimates of convenience sample-based experimental treatment effects in order to estimate what they might be in the population as a whole. The goal of population-based experiments is to address this problem through research design rather than analyses, combining the best aspects of both approaches, capitalizing on their strengths and eliminating many of their